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Abstract — We consider a relay channel where a relay helps 
the transmission of messages from one sender to one receiver. 
The relay is considered not only as a sender that helps the 
message transmission but as a wire-tapper who can obtain some 
knowledge about the transmitted messages. In this paper we 
study the coding problem of the relay channel under the situation 
that some of transmitted messages are confidential to the relay. 
A security of such confidential messages is measured by the 
conditional entropy. The rate region is defined by the set of 
transmission rates for which messages are reliably transmitted 
and the security of confidential messages is larger than a 
prescribed level. In this paper we give two definition of the rate 
region. We first define the rate region in the case of deterministic 
encoder and call it the deterministic rate region. Next, we define 
the rate region in the case of stochastic encoder and call it 
the stochastic rate region. We derive explicit inner and outer 
bounds for the above two rate regions and present a class of relay 
channels where two bounds match. Furthermore, we show that 
stochastic encoder can enlarge the rate region. We also evaluate 
the deterministic rate region of the Gaussian relay channel with 
confidential messages. 

Index Terms — Relay channel, confidential messages, informa- 
tion security 

I. Introduction 

The security of communication systems can be studied from 
a information theoretical viewpoint by regarding them as a 
kind of cryptosystem in which some messages transmitted 
through communication channel should be confidential to 
anyone except for authorized receivers. The security of a 
communication system was first studied by Shannon [1] from 
a standpoint of information theory. He discussed a theoretical 
model of cryptosystems using the framework of classical one 
way noiseless channels and derived some conditions for secure 
communication. Subsequently, the security of communication 
systems based on the framework of broadcast channels were 
studied by Wyner [2] and Csiszar and Korner [3]. Maurer [4], 
Ahlswede and Csiszar[5], [6], Csiszar and Narayan [7], and 
Venkatesan and Anantharam [8] studied the problem of public 
key agreements under the framework of multi-terminal channel 
coding systems. 

Various types of multiterminal channel networks have been 
investigated so far in the field of multi-user information theory. 
In those networks some kind of confidentiality of informa- 
tion transmitted through channels is sometimes required from 
the standpoint of information security. In this case it is of 
importance to analyze the security of communication from 
a viewpoint of multi-user information theory. The author [9] 
discussed the security of communication using relay channels. 
The author posed and investigate the relay channel with 
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confidential messages, where the relay acts as both a helper 
and a wire-tapper. Recently, Liang and Poor [10] studied the 
security of communication using multiple access channel by 
formulating and investigating the multiple access channel with 
confidential messages. 

In this paper we discuss the security of communication for 
relay channel under the framework that the author introduced 
in [9]. In the relay channel the relay is considered not only 
as a sender who helps the transmission of messages but as 
a wire-tapper who can learn something about the transmitted 
messages. The coding theorem for the relay channel was first 
established by Cover and El Gamal [11]. By carefully checking 
their coding scheme used for the proof of the direct coding 
theorem, we can see that in their coding scheme the relay helps 
the transmission of messages by learning all of them. Hence, 
this coding scheme is not adequate when some messages 
should be confidential to the relay. 

The author [9] studied the security of communication for 
the relay channel under the situation that some of transmitted 
messages are confidential to the relay. For analysis of this 
situation the author posed the communication system called 
the relay channel with confidential messages or briefly said 
the RCC. In the RCC, a sender wishes to transmit two 
different types of message. One is a message called the 
common message which is sent to the receiver and the relay. 
The other is a message called the private message which is 
sent only to the receiver and is confidential to the relay as 
much as possible. The knowledge that the relay gets about 
private messages is measured by the conditional entropy of 
private messages conditioned by channel outputs that the relay 
observes. The author [9] defined the rate region by the set of 
transmission rates for which common and private messages 
are transmitted with arbitrary small error probabilities and 
the security of private message measured by the conditional 
entropy per transmission is larger than a prescribed level. The 
author [9] derived an inner bound of the capacity region of 
the RCC. 

In this paper we study the coding problem of the RCC. 
In general two cases of encoding can be considered in the 
problem of channel coding. One is a case where deterministic 
encoders are used for transmission of messages and the other 
is a case where stochastic encoders are used. In the definition 
of the rate region by the author [9], deterministic encoders 
are implicitly assumed. In this paper we also consider the 
case of stochastic encoders. We define the rate region in the 
case where deterministic encoders are used for transmission 
and call it the deterministic rate region. We further define 
the rate region in the case of stochastic encoders and call 
it the stochastic rate region. We derive explicit inner and 
outer bounds for the above two rate regions and present a 
class of relay channels where inner and outer bounds match. 
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Furthermore, we give another class of relay channels, where 
the outer bound is very close to the inner bound. We also 
compare the results on stochastic and deterministic rate region, 
demonstrating that stochastic encoder can enlarge the rate 
region. We also study the Gaussian RCC, where transmissions 
are corrupted by additive Gaussian noise. We evaluate the 
deterministic rate region of the Gaussian RCC and derive 
explicit inner and outer bounds. We show that for some class 
of relay channels those two bounds match. 

Recently, Liang and Veeravalli [12] and Liang and Kram- 
mer [13] posed and investigated a new theoretical model of 
cooperative communication network called the partially/fully 
cooperative relay broadcast channel(RBC). A special case of 
the partially cooperative RBC coincides with the RCC in a 
framework of communication. However, in the problem setup, 
there seems to be an essential difference between them. The 
formulation of problem in the RBC is focused on an aspect 
of cooperation in relay channels. On the other hand, the 
formulation of problem by the author [9] is focused on an 
aspect of security in relay channels. Cooperation and security 
are two important features in communication networks. It is 
interesting to note that both cooperation and security simulta- 
neously occur in relay communication networks. 

II. Relay Channels with Confidential Messages 

Let X, S, y, Z be finite sets. The relay channel dealt with 
in this paper is defined by a discrete memoryless channel 
specified with the following stochastic matrix: 



(1) 



Let X be a random variable taking values in X and X" = 
X1X2 ■ ■ ■ Xn be a random vector taking values in A"". We 
write an element of A"" as a; X1X2 ■ ■ ■ Xn- Similar notations 
are adopted for S, Y, and Z. 

In the RCC, we consider the following scenario of com- 
munication. Let Kn and A/„ be uniformly distributed random 
variables taking values in message sets /C„ and Mn, respec- 
tively. The random variable Mn is a common message sent to 
a relay and a receiver. The random variable A'„ is a private 
message sent only to the receiver and contains an information 
confidential to the relay. A sender transforms /v„ and A/„ 
into a transmitted sequence X" using an encoder function /„ 
and sends it to the relay and the receiver. For the encoder 
function /„, we consider two cases; one is the case where /„ 
is deterministic and the other is the case where /„ is stochastic. 
In the former case /„ is a one to one mapping from /C„ x A^„ 
to A"". In the latter case /„ : /C„ x A^„ A"" is a stochastic 
matrix defined by 

/„(fc,TO) = {fn{x\k,m)}a;eX"Ak,rn) e /C„ x Mn ■ 

Here, fn{x\k, m) is the probability that the message (fc, m) is 
encoded as a channel input x. Channel inputs and outputs at 
the ith transmission is shown in Fig. [T] At the ith transmission, 
the relay observes the random sequence Z'^^ = [Zi, Z2, 
■ ■ ■ , Zi-i) transmitted by the sender through noisy channel, 
encodes them into random variable Si and sends it to the 
receiver. The relay also wishes to decode the common message 
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Fig. 1. Channel inputs and outputs at the ith transmission. 
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Fig. 2. Transmission of messages via relay channel using 

i'n, •Pn)- 



from observed channel outputs. The encoder function at the 
relay is defined by the sequence of functions {gi}"^i- Each 
gi is defined by gi : S. Note that the channel input 

Si that the relay sends at the ith transmission depends solely 
on the output random sequence that the relay previously 
obtained as channel outputs. The decoding functions at the 
receiver and the relay are denoted by ipn and ipn, respectively. 
Those functions are formally defined by ipn '■ 3^" K-n x 
M.n , ■ -2" Ain ■ Transmission of messages via relay 
channel using (/„„ ipn, Vn) is shown in Fig.|2] When 

/„ is a deterministic encoder, error probabilities of decoding 
for transmitted pair (fc, m) € /C„ x A^„ are defined by 

n 

Aj"'(fc,m)= ^ W_T{y^,Zi\xi{k,m),g{z'-~'^)) , 

{y,z): i=l 
^„(y)#(fc,m) 

n 

•^2"''("^) X! W'^{yr^Zi\xi{k,m),g{z'-^'^)) , 

iv,z): i=l 

where Xi{k,m) is the ith component of a; = /„(fc,m). The 
average error probabilities A^""* and Aj"' of decoding are 



defined by 
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1 



A, 



' " ' (fc,m)e/C„x7M„ 



\M 



(3) 



meMr. 



where |/C„| is a cardinality of the set /C„. When /„ is a 
stochastic encoder, error probabilities of decoding for trans- 
mitted pair {k,m) S /C„ x A^„ are defined by 

fi']"\k,m) 



A 



X] Y[^{yti^t\Mk,m),g{z' '^)) fn{x\k,m) , 

fJ-2 ('^) 



ix,y,z): 1=1 
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n 

^ E \{Y{y.,^A^.{k,m),g{z'-^))Ux\k,m). To state our result on an outer bound of 7^d(^), set 



The average error probabilities /i^""* and /Xj"' of decoding are 
defined by 



(n) A 

Mi = 



(ri) A 

= 



1 
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(^["\k,m), (4) 

(fc,m)eK;„xA1„ 



\Mr. 



(5) 



A triple (i?o, i?o) is achievable if there exists a sequence 
of quadruples {(/„, {.gj}r=i, V'n,'Pn)}^=i such that 



lim aJ"' = lim A 



0, 



1 



lim -log|7W„| = Ro, 

n — >oo n 

lim ilog|/C„| = 

n — >cxD fi 

lim -i?(/Y„|Z") > i?c. 

n — >oo 77, 

The set that consists of all achievable rate triple is denoted 
by TZd(r), which is called the deterministic rate region of the 
RCC. The definition of the stochastic rate region TZsCT) of the 

(n) (71) 

RCC is obtained by replacing Xl and Aj in the definition 
of 7?.d(r) by /z^"'' and fi'^^K respectively. 

III. Main Results 

In this section we state our main results. Proofs of the results 
are stated in Section VI. 



A. Deterministic Case 

In this subsection we state our results on inner and outer 
bounds of TZd{T). Let U be an auxiliary random variable 
taking values in finite set U. Define the set of random triples 

(f/, X, S) eU xX xS by 

Vi={{U,X,S): |W| < 1^-1151+ 3, 

U ^ XS ^YZ} , 

where U XS YZ means that random variables 
U, {X, S) and (Y, Z) form a Markov chain in this order. Set 

^d"^(r) ~ {{Ro, Rl,Re) '■ Ro,Ri,Re > , 

Ro < min{/(y; US), I{Z] U\S)} , 
Ri < IiX;Y\US), 
Rc 1^ Ri , 

Ro < [I{X;Y\US)-I{X;Z\US)]+ , 
for some {U, X, S) e Vi .} , 

where [a]+ = max{0,a}. Oohama [9] obtained the following 
result. 

Theorem 1 (Oohama [9]): For any relay channel F, 

7tf^(r) c7^d(F). 



^d°"*Hr) = {(i?o,i?i,i?c) :i?o,i?i,i?c >0, 

Ra < min{/(y; US),I{Z; U\S)} , 
Ri < I{X;YZ\US), 
Ro + Ri< I{XS; Y) , 
Rc 1^ Ri , 

Re < I{X;Y\ZUS), 
for some (U, X, S) e Vi .} . 

Then, we have the following theorem. 
Theorem 2: For any relay channel F, 

7^d(^) c7^|l°"'^(F). 

An essential difference between inner and outer bounds of 
TZd{T) is a gap A given by 

A = I{X; Y\ZUS) - [I{X; Y\US) - I{X; Z\US)] 
= I{X; ZY\US) - I{X; Y\US) 
^ I{X;Z\YUS) . 

Observe that 

A = H{Z\YUS) - H(Z\YXUS) 

= H{Z\YUS) - H{Z\YXS) (6) 
< H{Z\YS) - H{Z\YXS) = I{X- Z\YS) , 

where (|6]l follows from the Markov condition U ^ XS 
YZ. Hence, A vanishes if the relay channel W = 

{r{z,y\x,s) }(x,s,y,z}exxsxyxz satisfies the following: 



F(z, y\x, s) = T{z\y, s)F(y|.T, s) 



(7) 



The above condition is equivalent to the condition that 
X, S, y, Z form a Markov chain X ^ SY ^ Z in this 
order Cover and El. Gamal [11] called this relay channel the 
reversely degraded relay channel. On the other hand, we have 

I{X;Y\ZUS) 
= H{Y\ZUS) - H{Y\ZXUS) 
<H{Y\ZS)-H{Y\ZXS)^I{X;Y\ZS), (8) 

where (|8]l follows from the Markov condition U XSZ 
Y. The quantity I{X\Y\ZU S) vanishes if the relay channel 
F satisfies the following: 



F(z, y\x, s) = T{y\z, s)F(z|a;, s). 



(9) 



Hence, if the relay channel F satisfies (|9]l, then R^ should be 
zero. This implies that no security on the private messages is 
guaranteed. The condition (|9]l is equivalent to the condition 
that X, S, Y, Z form a Markov chain X ^ SZ Y in this 
order Cover and El. Gamal [11] called this relay channel the 
degraded relay channel. Summarizing the above arguments, 
we obtain the following two corollaries. 

Corollary 1: For the reversely degraded relay channel F, 
we have 

7^fl'"^(F) =7^d(F) =:^fi°"*^(F). 
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Corollary 2: In the deterministic case, if the relay channel 
r is degraded, then no security on the private messages is 
guaranteed. 

Next, we derive another inner bound and two other outer 
bounds of TZd{T). Define a set of random triples {U,X, S) E 
U xX xS by 

V2 = {{U,X,S): \U\ < IZIIA-II^I+S, 
U XSZ ^ Y} . 

It is obvious that Vi C P2. For given (U, X,S) xX xS, 
set 

n{u,x,s\T) 

i?o < min{/(y; US), I{Z; U\S)} , 
Ro + Ri< I{X;Y\US) 

+ mm{I{Z;U\S),I{Y;US)}, 
Rc < Ri , 

Rc < [I{X; Y\US) - I{X; Z\US)]+ .} . 
Furthermore, set 

nf'\T)^ U 7^(c/,x,5|^), 

{u,x,s)eVi 
^(^out)^P^A y Ti(^u^x,S\T). 

{u,x,s)eV2 

Then, we have the following. 

Theorem 3: For any relay channel F, 

7^|^'"^(F) C 7^d(F) C 7^|l°"*^(F). 
Now we consider the case where the relay channel F 
satisfies 

Tiy,z\x,s) = r{y\x,s)T{z\x). (10) 

The above condition on F is equivalent to the condition that 
X, S, Y, Z satisfy the following two Markov chains: 



where we set 



A 



Y ^ XS ^ Z ,S ^ X 



z . 



The first condition is equivalent to that Y and Z are condi- 
tionally independent given SX and the second is equivalent 
to that Z and S are conditionally independent given X. 

We say that the relay channel F belongs to the independent 
class if it satisfies ( fTOl i. For the independent class of relay 
channels, we derive an outer bound of TZd{T). To state our 
result, set 



7^ 



(out) 



(r) 



A 



{{Ro, i?o) : Ra,Ri,Rc > , 

Ra < min{/(r; US),I{Z; U\S)} , 
Ro + Ri< I{X- Y\US) + [C(C/, S, Y, Z)] + 
+ min{/(Z; U\S),I{Y; US)} , 

Rc < Ri , 

Re < [I{X; Y\US) - I{X; Z\US) 
+C{U,S,Y,Z)]+, 
for some (U, X, S) e Vi .} , 



C(C/, S, Y, Z) " I{XS; Y\U) - I{XS] Z\U) 

-~[I{X;Y\US) ~ I{X;Z\US)] 
= I{S-Y\U)-IiS;Z\U) 
= H{S\ZU)~H{S\YU). 

The quantity (^{U, S, Y, Z) satisfies the following. 
Property 1: For any ([/, X, S) e V2, 

aU,S,Y,Z)<I{XS;Y\Z). 
Proof: We have the following chain of inequahties: 



(11) 



au,s,Y,z) 

= H{S\ZU)- H{S\YU) 

< H{S\ZU) - H{S\YZU) 
= IiS;Y\ZU) 

= HiY\ZU)- H{Y\ZUS) 

< H{Y\Z) - H{Y\ZUS) 

< H{Y\Z) - H{Y\ZXSU) 

= H{Y\Z) - H{Y\ZXS) = I{XS; Y\Z), (12) 
where the last equality follows from the Markov condition 

u zxs ^Y. m 

Our result is the following. 

Theorem 4: If F belongs to the independent class, we have 

7^d(^) c7^^°"')(F). 

B. Stochastic Case 

In this subsection we state our results on inner and outer 
bounds of TZs(r). Define two sets of random quadruples 

{Uy,X,S) e U xV xX xS hy 

Q, = {{U,V,X,S) : \U\ < 1^-1151+3, 

\V\<{\X\\S\)'+A\X\\S\+3, 
U ^ XS ^YZ , 
US ^ X} , 

Q2 = {{UV,X,S): \U\ < \Z\\X\\S\ + 3 , 

|V| <(|Z||A'||5|)V4|Z||A'||5|+3, 
U ^ V ^ XSZ ^Y , 
US ^VX ^ Z , 
US ^ X}. 

It is obvious that Qi C Q2. For given {U, V, X,S) e U xV 
xX xS, set 

TZiU,V,X,S\T) 

= {{Ro, Ri,Ro) ■ Ro, Ri,Rc > , 

Ro < min{/(y; US),I{Z- U\S)} , 
Ro + Ri< I{V;Y\US) 

+ mm{I{Y;US),I{Z;U\S)} , 

Rc < Ri , 

Rc < [I{V; Y\US) - I{V; Z\US)]+ .} . 
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Set 

7^f)(r)^ y 7^(t/,y,x,5|^), 

{u,v,x,s}eQi 

-^(out) A y ^^^^ ^1^^ ^ 

{u,v,x,s)eQ2 

Then, we have the following. 

Theorem 5: For any relay channel F, 

7^(''^'(^) ^7^s(^) c7^(°"')(F). 

Similarly to the deterministic case, we estimate the quantity 
I{X] Y\US) - I{X; Z\US). We have the following chain of 
inequalities: 

I{V;Y\US)-I{V;Z\US) 

< I{V] YZ\US) - I{V; Z\US) 
= I{V;Y\ZUS) 

= H{Y\ZUS) - H{Y\ZVUS) 

< H{Y\ZS) - H{Y\ZXVUS) 

= H{Y\ZS) - H{Y\ZXS) = I{X; Y\ZS) , (13) 

where (fTsT i follows from the Markov condition 

U ^ XSZ ^Y . 
Then, if F is degraded, for any {U,V, S, X) G Q2, we have 

KV; Y\US) - I{V; Z\US) < . 

Hence, if the relay channel F is degraded, then i?c should be 
zero. This implies that no security on the private messages is 
guaranteed for the degraded relay channel. Thus, we obtain 
the following corollary. 

Corollary 3: When the relay channel F is degraded, no 
security on the private messages is guaranteed even if /„ is a 
stochastic encoder. 

IV. Secrecy Capacities of the RCC 

In this section we derive an exphcit inner and outer bounds 
of the secrecy capacity region by using the results in the 
previous section. 

A. Deterministic Case 

We first consider the case where /„ is a deterministic 
encoder The secrecy capacity region Cds(F) for the RCC is 
defined by 

Cds(r) = {{Ro,Ri) : {Ro,Ri,Ri) e 7^d(F)}. (14) 
From Theorems [T] and |2] we obtain the following corollary. 
Corollary 4: For any relay channel F, 

c(f'(F)CCds(r)CC^"*)(F), 



where 

C£"'(r) = {{Ro,Ri):Ro,Ri>0, 

Ro < min{/(r; US), I{Z; U\S)} , 
-Ri < [HX; Y\US) - I{X; Z\US)]+ , 
for some (C/, X, S) eVi.}, 

Clr^(r) = {(i?o,i?i) :i?o,i?i >0, 

Ro < min{/(r; US), I{Z; U\S)} , 

Ri < I{X;Y\ZUS), 

for some {U, X, S) G Pi .} . 

In particular, if F is reversely degraded, we have 

c^':^(r)=Cd.(F)=clr'(r). 

From Theorem [3] we obtain the following corollary. 
Corollary 5: For any relay channel F, 

c!:\r)cc^,ir)cc[T\r), 

where 

C^r^(r) - {(i?o,i?i) :i?o,i?i >0, 

Ra < miii{/(r; US), I{Z; U\S)} , 
Ri < Y\US) - I{X; Z\US)\+ , 

for some ([/, X, S) G 7^2 ■} ■ 
From Theorem m we obtain the following corollary. 
Corollary 6: If F belongs to the independent class, we have 

Cds(r)cc(,°"')(F), 

where 

Clr^(r)= {(i?o,i?i):i?o,i?i>0, 

i?o < min{/(y; US), I{Z; U\S)} , 
Ri < Y\US) - I{X; Z\US) 

+aU,S,Y,Z)]+, 

for some {U,X,S) eVi.}. 
Now, we consider the special case of no common message. 
Set 

7^dlc(^) = {{Ri,Rc) ■■ (o,i?i,i?e) e 7^d(F)} 

and define the secrecy capacity by 

Cds(r) = max i?i = max i?i . 

(/?.i,fli)e7?,dio(r) (o.fli)eCd.(r) 

Typical shape of the region 7?.dio(r) and the secrecy capacity 
Cds(F) is shown in Fig. |3] 

From Theorems [T| and |2] we have the following corollary. 

Corollary 7: For any relay channel F, 

^!,Ti(^)c7^dlc(^)c^(°f (F), 
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Cds(r) 



Fig. 3. The region 7?.dic(r) and the secrecy capacity Cds(r). 



where 



Ri < I{X;Y\US), 
Rc < Ri , 

R, < [I{X; Y\US) - I{X; Z\US)]+ , 
for some {U,X,S) eVi.}, 



T^die\^) ^ {{Ri,Rc)-Ri,Rc>0, 



Ri < I{X;YZ\US), 
Rc 1^ Ri , 

Rc < I{X;Y\ZUS), 

for some iU,X,S) eVi.}. 



Furthermore, 



Then, we have the following. 

Corollary 8: For any relay channel F, 

7^iTi(^)c7^dlc(^)c<f (F). 

Furthermore, 

Cds(r) < max [I(X; YIUS) - I(X; ZIUS)]'^ . 

Finally we state a result which is obtained as a corollary of 
Theorem H Set 

^dTf (r) - {(i?i,i?c) :i?i,i?c>o, 

Ri<I{X-Y\US) + [aU,S,Y,Z)] + 
+ xmn{I{Y]US),I{Z-U\S)} , 
Rc < Ri , 

i?c < [I{X; Y\US) - I{X; Z\US) 
+aU,Y,S, Z)]+, 
for some {U,X,S) (zVi .} 

Then we have the following. 

Corollary 9: If F belongs to the independent class, we have 



7^dle(^)c7^^°f (F). 



Furthermore, 



max \I{X]Y\US) I{X-Z\US)Y 

iU,X,S)GVi 

< Cd.(F) 

< max I(X:Y\ZUS). 

{U,X,S)&Vi 

In particular, if F is reversely degraded, we have 

^d^i(^) = 7^dle(^) = 7^(r'(^) 

and 

Cds(F) = max [I(X; YlUS) - I(X; ZIUS)] . 
iu,x,S)eVi 

Next, we state a result which is obtained as a corollary of 
Theorem [3] Set 



A 



and 



TZie{U,X,S\T) 

n{U,X,S\T)n{{Ro,RuRc) : i?o = 0} 
= {(^i,^c) : Ri,Rc > 0, 
Ri < I{X;Y\US) 

+ mm{I{Y;US),I{Z;U\S)}, 
Rc 1^ Ri , 

i?e < [I{X; Y\US) - I{X; Z\US)]+ .} 

^dTi(r)= U n^,iu,x,s\r), 

(u,x,s)ePi 

niT\T)^ y ^z^,{u,x,s\^). 

{u,x,s)eV2 



Cds(r) < max \I(X; Y\US) - I(X; ZlUS) 

(U,X,S)eVi 

+C{U,S,Y,Z)]+ 

= max \IiXS;Y\U)- I{XS;Z\U)]^ 
{u,x,s)eVi 



B. Stochastic Case 

The stochastic secrecy capacity region Css(F) for the RCC 
is defined by 

CssiT) = {(i?o, Ri) : (i?o, Ri,Ri) G TZsiT)} . (15) 
To describe our result set 

CsiU,V,X,S\T) 

= niU,V,X,S\T)n{iRa,Ri,R,) : Ri ^ Rc} 

= {{Rq, Ri) ■ Ro, Ri >0 , 

Ra < min{/(r; US), I{Z; U\S)} , 
Ri < [I{V;Y\US)~I{V;Z\US)]+} 

and 

A 



CirHF)^^ y C.iU,V,X,S\T), 

(u,v,x,s)eQi 

cirHr)= y c.(c/,y, x,5|F). 

(U,V,X,S)<=Q2 

From Theorem |5] we obtain the following corollary. 
Corollary 10: For any relay channel F, 

cirnr)cCss(F)cci°"*)(F). 

In particular, if F is degraded, we have 

cir'(r) = c..(F) = cir)(F). 
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Next, set 

7^,le(^) = {(o,i^l,i?e)e7^s(^)} 

and define the secrecy capacity by 

Css(r) = max i?i = max Ri . 

{Ri,Ri)en,i^(r) (o,i?i)ec,,(r) 

To describe our result, set 

ni,{u,v,x,s\r) 

= 7^([/, V, X, S\T) n {{Ro, Ri,R,) : i?o = 0} 
= {(i?i,i?c) : Ri,Rc > 0, 

-Ri < /(yirit/s*) 

+ miii{/(y;C/S'),/(Z;[/|5)}, 
i?o < [/("t^; ^1^7^) - I{V: Z\US)]+ .} 



and 



,(out) 



A 



A 



y 7^le(f/,y,x,5|^), 

((7,V,X,S)eQi 

7^^r^(r)= U 7^le(c/,^/,^,5|^). 

[U,V,X,S)£Q2 

From Theorem |5] we have the following corollary. 
Corollary 11: For any relay channel F, 



7^ii"^(^)c7^.le(^)c7^^-^(^). 



(out) , 



Furthermore, 



max [/(F;y|[/S')-/(F;Z|[/S')]+ 

{U,V,X,S)£Qi 

< max [/(F;y|C/5)~J(F;Z|(75)]+ 

(u,v,x,s)eQ2 



V. Gaussian Relay Channels with Confidential 
Messages 

In this section we study Gaussian relay channels with con- 
fidential messages, where two channel outputs are corrupted 
by additive white Gaussian noises. Let (^1,^2) be correlated 
zero mean Gaussian random vector with covariance matrix 



\p\<l. 



Let {(^1,1, $2,i)}i^i be a sequence of independent identically 
distributed (i.i.d.) zero mean Gaussian random vectors. Each 
(Ci,ij?2.i) has the covariance matrix S. The Gaussian relay 
channel is specified by the above covariance matrix E. Two 
channel outputs Yi and Zi of the relay channel at the ith 
transmission are give by 

Yi ^ Xi + Si + ^i^i , 

Zi ^ Xi + ^2,i ■ 

Since (^i.i, ^2.4)1 * — 1:2,- • -,71 have the covariance matrix 
S, we have 

£.2.i — P\ TTCl.i + ?2|l,i , 
V Jvi 



(16) 



where S,2\i.i,i = I7 2, • • • , n are zero mean Gaussian random 
variable with variance (1 — p'^)N2 and independent of ^1 ,;. In 

particular if S satisfies A^i < N2 and p = ^J^, we have for 
i = 1, 2, • • • , n, 

= x, + 5, + ei,,:, 

Zi ~ Xi + £_i i + ^2|i,-i 

which implies that for i ~ 1,2, • • • , n, Zi {Yi,Si) — > 
Xi. Hence, the Gaussian relay channel becomes reversely 
degraded relay channel. Two channel input sequences {XiY^L^ 
and {Sij'^^i are subject to the following average power 
constraints: 

'-Y.^[x!]<p,/-Y.^[sf]<P2. 

Let 7?.d(Pi, P2IS) be a rate region for the above Gaussian 
relay channel when we use a deterministic encoder /„. To 
state our result set 



A 



7^f^(Pl,P2|s) 

{(i?o, Ri,Rc) ■ Ro, Ri,Rc>0 : 



Rq < max mill 

0<?)<1 



N2)] 



Rc ^ Ri , 
Ro < 



r ( ^\ r ( 

\Ni ) \ N2 



for some < 6* < 1 .} , 



7^f,°"*^(Pl,P2|s) 

~ {{Ro, Ri, Rc) ■ Ro,Ri,Rc > 0, 

Ro<unniC' ^'^^+^-+^^^^ 

C 



ePi+Ni 



Ro 



Ri<C 



Ro < Ri 



ePi+N- 
ePi 



Rr < 



where C{x) = ^ log(l + x) . Our result is the following. 
Theorem 6: For any Gaussian relay channel, 

7^f ' (Pi , P2 1 s) c 7^d(Pl , P2 |s) c 7^|^°"') (Pi , P2 1 e) . (i?) 

In particular, if the relay channel is reversely degraded, i.e., 
iVi < N2 and p = ,/^, then 



C 



(l-p 




N-1 + N2 


-2p^NiN2 / 


P1+P2- 


|-2-^er7PiP2 




Ni 


( „ 


ePi 







-I + 



\ iVi+lV2-2pVNlN2 / 

for some 0<6'<1,0<77<1.}. 



7^^l'"^ (Pi , P2 1 s) - 7^d (Pi , P2 1 s) = nr^' (Pi , P2 1 s) 



(out) , 
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Proof of the first inclusion in ( fTTj ) in the above theorem is 
standard. The second inclusion can be proved by a converse 
coding argument similar to the one developed by Liang and 
VeeravaUi [12]. Proof of Theorem |6] is stated in the next 
section. 

Next, we study the secrecy capacity of the Gaussian RCCs. 
Define two regions by 



A 



Cds(Pl,P2|S) 

{(i?0,i?l) : (i?o,i?l,i?l) e7^d(^'l,^'2|S)} , 

7^dlc(Pl,P2|s) 



A 



{(i?l,i?c) : (0,i?i,i?e) e7^d(Pl,P2|S)} . 
Furthermore, define the secrecy capacity Cds(-Pi, -P2IS) by 



Cd.(Pi,P2|s; 



A 



max Ri 

(fli,fli)GKdio(Pi,P2|S) 

max Ri 

(o,fli)eCd.(Pi,P2|s) 



We obtain the following two results as corollaries of Theorem 

m 

Corollary 12: For any Gaussian relay channel, we have 

c^^:^ (Pi , P2 1 s) c Cds (Pi , P2 1 s) c 4:"') (Pi , P2 1 s) , 

where 

C(f'(Pl,P2|S) 

= {(Po,Pi) : Po,Pi > 0, 



Rn < max mill i C 

0<?)<1 



ePi+P2+2y/eflPiP2 



ePi+Ni 



Ri < 



"-^ \ePi+N2 J j 
for some < 6* < 1 .} . 



— {(P07 Pi) : Ro, Ri >0 , 

T3 ^ ■ \ n ( 9Pl+P2+2jSf,PlP2 

^0 - o<,f<i"'''' r V 



\8Pi+N2 J / 



Pi < 



SPi 



(l-p2)NiJV2 



\ JVi + JV2-2p^iViJV2 / 

for some < < 1 .} . 



In particular, if iVi < N2 and p = y ^^^^ 

4l:^)(Pi,P2|E) =Cds(Pi,P2|S) -C^r)(Pi,P2|E). 
Corollary 13: For any Gaussian relay channel, we have 



7^^t^(Pl,P2|s) c 7^dlc(Pl,P2|s) c 7^^7;^(Pl,P2|s) , 



(out) , 



where 

7^^l^i(Pl,P2|s) = {(Pi,Pe) : Pi,Pc > 0^ 

Ni 



Rc < Pi 



Pc < 



A 



7^!^7e*^(Pl,P2|S) 
{(Pi, Pc) : Pi, Pc > , 

\ lVi + lV2-2p^iViJV2 



Pc < Pi , 



Pc < 

Furthermore, 



Pi 

(l-p^)JVl~ 



Afl+Af2-2p^Afl Af2 



<Cd.(Pl,P2|S) 



< 



c 



(1- p'^)NiN2 



C 



we have 

1\2 



\ JVi+JV2-2p^iVi Af2 

In particular, if A^i < iV2 and p = 

7^(\"i(Pl, P2II]) = 7^dlc(Pl, P2IS) = Tzi\f{PuP2\^) , 

and 

Cds(Pi,P2|S) = c(|^) -c(^) . 
Note that the secrecy capacity C'ds(Pi, P2IS) for the re- 
versely degraded relay channel does not depend on power 
constraint P2 at the relay. This implies that the security of 
private messages is not affected by the relay. Leung-Yan- 
Cheong and Hellman [14] determined the secrecy capacity for 
the Gaussian wire-tap channel. The above secrecy capacity is 
equal to the secrecy capacity of the Gaussian wire-tap channel 
derived by them. 

VI. Proofs of the Theorems 

In this section we state proofs of Theorems [T]|6] stated in 
the sections III and V. 

In the first subsection we prove Theorem [T] the inclusion 
7^^'"^(^) C 7^d(^) in Theorem m and the inclusion 7^^'"'(^) 
C 7?,s(r) in Theorem |5] In the second subsection we prove 
Theorem 121 the inclusion 7^d(^) C 7^|^°"*^(^) in Theorem |3] 
and the inclusion 7?.s(r) C 7?.i°"*'(r) in Theorem |5] Proof of 
Theorem |6] is given in the third subsection. 

A. Derivations of the Inner Bounds 

We first state an important lemma to derive inner bounds. 
To describe this lemma, we need some preparations. Let 7^, 
Jn, and Cn be three message sets to be transmitted by the 
sender Let Tn,Jn, and i„ be uniformly distributed random 
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/o = mm{I(Y;US), I{Z;U\S)} 
/0+/1/--" Ii = IiX;Y\US), h=I{X;Z\US) 

Fig. 4. Shape of the region TZ{U, X, S\r). 



variable over Tn, Jn and £„ respectively. Elements of %i 
and Jn are directed to the receiver and relay. Elements of 
£„ are only directed to the receiver Encoder function /„ is 
a one to one mapping from T^jX j7„x /C„ to A"". Using the 
decoder function 1])^, the receiver outputs an element of T^x 
i/nX K-n from a received message of 3^". Using the decoder 
function (p„, the relay outputs an element of T^jX J7ii from a 
received message of Z". Formal definitions of ?/;„ and (p„ are 
-0" : ^ ^ X Jn X £„ , ifn : Z" ^ T;^ X J„ . We define 
the average error probability of decoding at the receiver over 
T^x j7nX Ln in the same manner as the definition of A$^"' 
and use the same notation for this error probability. We also 
define the average error probability of decoding at the relay 
over x in the same manner as the definition of Ag"' and 
use the same notation for this probability. Then, we have the 
following lemma. 

Lemma 1: Choose ([/, X, S) G Vi such that I{X; Y \YS) 
> I{X; Z\YS). Then, there exists a sequence of quadruples 

V'n,'^n)}^=l SUCh that 



lim aJ"' = lim A^"' , 
lim - \og\%,\ ^ mm{IiY; US), I iZ;U\S)}, 

n—*oo Tl 

lim -\og\Jn\^ I{X;Z\US), 

lim - log \Cn\ = I{X- Y\US) - I{X- Z\US) , 

n — >-oo 77, 

lim -H{Ln\Z") > I{X; Y\US) - I{X; Z\US) . 



An) 



The above lemma is proved by a combination of two coding 
techniques. One is the method that Csiszar and Korner [3] used 
for deriving an inner bound of the capacity regions of the 
broadcast channel with confidential messages and the other 
is the method that Cover and El Gamal [11] developed for 
deriving a lower bound of the capacity of the relay channel. 
Outline of proof of this lemma is given in Appendix A. 



Proof of n'i"\T) C 7^d(^).• Set 



A 



Io=min{I{Y;US),IiZ;U\S)}, 
h = I{X; Y\US), h ^ I{X- Z\US) 



Reciever 
Relay 



iiiin{/(7;;75),/(Z;C/15)} 



I{X;Y\US) 



I(X;Z\US) 



Bits of Common 
Messages 



mil 



Security 
Level 



Bits of Private 
Messages 



Fig. 5. Information contained in the transmitted messages. 

We consider the case that 1\ > l2- The region TZ{U, X, S\r) 
in this case is depicted in Fig.|4] From the shape of this region 
it suffices to show that for every 

ae [0,mm{I{Y;US)J{Z;U\S)}], 

the following {Ro, Ri, R^) is achievable: 

Ro = min{/(r; US),I{Z; U\S)} ~ a , 
Ri = I{X;Y\US)+a, 
i?c = HX; Y\US) - I{X; Z\US) . 
Choose 71' and 71" such that 



T — T' -x T" 



1 



lim - log 17;; I 

n — 'oo Ji 



min{/(y; US), I{Z] U\S)} - a . 
We take 

A^n — 7^i 1 ^71 — 7^ X J7n X Cji . 

Then, by Lemma [U we have 



lim A 



(») 



lim A 



(") 



0, 



n—^oo 71 — >-oo 

lim -log|/C„| = /(X;y|[/S') +a, 

n — >-oo fi 

lim -log|7W„| =mm{I{Y;US),I{Z;U\S)}-a, 

n — >OQ Ji 

lim ii7(A'„|Z") > lim ii7(L„|Z") 

> IiX;Y\US) - IiX;Z\US) . 

To help understating the above proof, information quantities 
contained in the transmitted messages are shown in Fig. |5] ■ 

Proof of TheoremU} Since ul^'^^T) C 7^|^'"^(^), we have 
Theorem [T] ■ 

Proof of ni'^'^T) C UsiT): Choose iU,V,X,S) G Qi. 
The joint distribution of {U, V, X, S) is given by 

Puvxs{u,v,x,s) 
= Pusviu, s, v)px\vix\v) , {u,v,x,s) eU xV X X X S . 

Consider the discrete memoryless channels with input alphabet 
V X 5 and output alphabet y x Z, and stochastic matrices 
defined by the conditional distribution of {Y, Z) given V, S 
having the form 

r(y, z\v, s) = ^ r(y, z\x, s)px\vix\v) . 



10 



Any deterministic encoder /'^ : /C„ x V" for this new 

RCC determines a stochastic encoder /„ for the original RCC 
by the matrix product of //^ with the stochastic matrix given by 
Px\v = {Px\v{^W)}{v.x)eVxx- Both encoders yield the same 
stochastic connection of messages and received sequences, 
so the assertion follows by applying the result of the first 
inclusion in Theorem |3] to the new RCC. ■ 
Cardinality bounds of auxiliary random variables in Vi and 
Qi can be proved by the argument that Csiszar and Korner 
[3] developed in Appendix in their paper. 

B. Derivations of the Outer Bounds 

In this subsection we derive the outer bounds stated in 
Theorems [2]|5] We first remark here that cardinality bounds of 
auxiliary random variables in V2 and Q2 in the outer bounds 
can be proved by the argument that Csiszar and Korner [3] 
developed in Appendix in their paper. 

The following lemma is a basis on derivations of the outer 
bounds. 

Lemma 2: We assume (i?o, -Ri, -Rc) is achievable. Then, we 
have 

nRo < min{/(r"; Af„), /(Z"; Af„)} + 
nRi<I{Kn;Y^\M„)+nS2^n 
n{Ro + Ri) < I{Y"; i^„M„) + nJg.n 
nRc < nRi + nS^^n 

nR, < I{Kn; Y^\M„) - I{K,,- Z^\M^) + nS^^n 

where {^i.nj^i, * = 1,2,3,4,5 are sequences that tend to 
zero as n 00. 

Proof: The above Lemma can be proved by a standard 
converse coding argument using Fano's Lemma. We omit the 
detail. A similar argument is found in Csiszar and Korner [3] 
in Section V in their paper ■ 

We first prove 7^d(^) C 'R.'-°''^\r) . From Lemma |2l it 
suffices to derive upper bounds of 

/(Z"; M„), /(F"; M„), /(K„; r"|M„), 

J(y"; KM,I{K^; r"|M„) - /(X„; Z"|M„). 

For upper bound of the above five quantities, we have the 
following Lemma. 

Lemma 3: Suppose that /„ is a deterministic encoder. Set 

U.,=MnY'-^Z'-\ z = 1,2, 

For i = 1,2, ■■■ ,n, Ui, XiSi, and Y^Zi form a Markov chain 
Ui ~* XiSi YiZi in this order. Furthermore, we have 

n 

/(r";M„) <^/(K,;[/,5,), (18) 

i=l 
n 

I{Z'^;M^)<Y,I{Z,-U,\S{), (19) 

n 

I{Y"; KnMn) < ^ I{Y; X,S,) , (20) 

4=1 

n 

I{K„;Y"\M^) <Y,I{X,;Y,Z,\UA), (21) 



/(X„;y"|M„)-/(if„;Z"|M„) 

n 

<Y,I{X,;Y,\Z,UA). (22) 



1=1 



Proof of Lemma [3] is given in Appendix B. 

Proof of Theorem^ We assume that (i?o, ^1, Rc) is achiev- 
able. Let Q be a random variable independent of K„ MnX^Y^ 
and uniformly distributed over {1, 2, • • • , n}. Set 



Y ^ Y Q ^ Q V — V 7—7 



Furthermore, set 



U = VqQ ^ Z^-^Y^-HUQ . 



(23) 



(24) 



Note that UXSYZ satisfies a Markov chain U XS YZ. 
By Lemmas |2] and |3] we have 

i?o < min{/(y; US\Q), I{Z; U\SQ)} + 5i,„ 
< min{/(y; US),I{Z; U\S)} + <5i.„ 

Ri < I{X;YZ\US) + S2,n 
RQ + Ri<IiXS;Y\Q)+S3,„ 

<I{XS;Y)+S3^n 

Rc < Rl + S4,n 

Rc < I{X;Y\ZUS) + S5,n. 

'(25) 

Using memoryless character of the channel it is straightfor- 
ward to verify that U XS YZ and that the conditional 
distributions of and given coincide with the corresponding 
channel matrix. Hence by letting n cx) in (IZST i, we obtain 

(i?o,i?i,i?o)e7l(°"*)(r). ■ 

Next, we prove the inclusions 7?.d(r) C TZ^^"^\t) and 
^d(r) Q 7t|[°"*'' (r). From Lemma |2] it suffices to derive 
upper bounds of the following five quantities: 

/(Z";Af„),/(r";M„), 

y"|M„) + I(Y"; A/„) = I{Y''-KM , 
I{Kn; r"|A//„) + /(Z"; A//„) , (26) 
/( A-„ ; y" I A//„ ) - I{K\ ; Z" | M„) . (27) 

Since 

/(i^„;y"|A//„)+/(Z";A//„) 
- I{K„; y"|M„) - Z"|A^„) + /(A-„A//„; Z") , 

we derive an upper bound of ( |26] l by estimating upper bounds 
of /(/•«:„ Af„;Z") and 

The following two lemmas are key results to derive the outer 
bounds. 

Lemma 4: Suppose that /„ is a deterministic encoder Set 

t/, = y/|iZ'-iA//„ , z=l,2,---,n. 
For i = 1, 2, • • • , n, Ui, XiSiZi, and Yi form a Markov chain 
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Ui XiZiSi Yi in this order. Furthermore, we have 

n 

I{Y^-,M^)<Y,I{Y,-U,S,), (28) 

n 

I{Z''-Mr,)<Y,I{Z,-U,\S,), (29) 

i=i 

n 

i{Y"; K,Mn) < Hy.;X,UA) , (30) 

n 

I{Z'^- K^Mn) < ^ I{Z,; X,U,\S,) , (31) 

i=l 

/(if„;F"|M„)- J(ir„;Z"|A/r„) 

71 

< Y,\UA) - I{Xf, Z,\UA)} . (32) 

1=1 

Lemma 5: Suppose that /„ is a deterministic encoder Set 

U, = Y'-^Z'^:^^Mn, i = l,2,---,n. 

For i = 1,2, ■■■ ,n, Ui, XiSiZi, and Yi form a Markov chain 
Ui XiZiSi Yi in this order. Furthermore, we have 



(33) 
(34) 
(35) 
(36) 



/(r";Af„) <^/(i;;;t/,5,), 

1=1 

n 

/(Z";A/„) <^/(Z,;f/,|5,), 
i=i 

n 

/(r"; i^„Af„) < I{Yf,X,UA) , 

i=l 
n 

I{Z^; K,M < Y Ii.Z,; X,U,\S,) , 

i=l 

I{K^;Y'-\M,,) ~ I{K,,;Z''\M^) 

n 

< J2 {HX^S^■, y,|C/,) - I{XA; z,\u,) 

i=i 

+iiUf,z,\XA)} 

n 

= Y {I{X^;Y,\UA) - I{X,; Z,\UA) 

4=1 

+C{U,, S,, Y,, Zi) + I{U,; Z,\XA)} . (37) 
Proofs of Lemmas |4] and |5] are given in Appendixes D and 

E, respectively. 

Proof of UdiT) C 7^|^°"'^(^).• We assume that (i?o, Ri,Rc) 

is achievable. Let Q, X, Y, Z, S be the same random variables 

as those in the proof of Theorem |2] Set 



u^UqQ = y^^^zQ-Hu,q. 



(38) 



Note that UXSYZ satisfies a Markov chain U XSZ Y. 
By Lemmas |2] and |4] we have 

i?o < min{/(r;t/^),/(Z;[/|S')} + (5i,„ ' 
Ro + Ri < I{X;Y\US) 

+ mm{I{Y; US),I{Z; U\S)} + h.n \ (39) 

i?c < -Rl + 1^4, n 

i?e < I{X; Y\US) - I{X; Z\US) + S^^n , . 



where ^3,„ = max{Ji^„ + S2.n,S3,n} ■ By letting n cx) in 
we conclude that iRo,Ri, Re) e 7?.^°''*^(r). ■ 
Proof of TZdiT) C 7t,^°"*'(r).- We assume that {Rq, Ri,Re) 
is achievable. Let Q, X, Y, Z, S be the same random variables 
as those in the proof of Theorem |2] We set 



u = UqQ^ yQ-'z^+,m, 



(40) 



Note that UXSYZ satisfies a Markov chain U XSZ Y. 
Furthermore, if F belongs to the independent class, we have 

Z^XS^Y, U^XS^Z, (41) 

which together with U XSZ Y yields 

U XS ->YZ . 

By Lemmas |2] and |5] we have 

Ro < mm{I{Y; US), I{Z; U\S)} + 5i,n 
Ro + Ri< I{X; Y\US) + [C(C/, S, Y, Z)]+ 

+ nim{I{Y;US),I{Z;U\S)}+~5^,, ) (42) 

< ^1 + ^4,n 

i?e < I{XS; Y\U) ~ I{XS; Z\U) + Jg,,. ■ 

Note here that the quantity I{U; Z\XS) vanishes because of 
the second Markov chain of ( l4Tl i. By letting n ^ oo in ( |42] |. 
we conclude that (i?o, i?i, Rc) & 7?,^°"*'(r). ■ 

Finally we prove 7?.s(r) C 7?.s°" (F). The following is a 
key result to prove the above inclusion. 

Lemma 6: Suppose that /„ is a stochastic encoder. Let Ui, 
i = l,2,---,nbe the same random variables as those defined 
in Lemma m We further set Vi = UiSiKn- For i = 1,2, ■ ■ ■ ,n, 
UiViXiSiZi satisfies the following Markov chains 

Ui — > Vi — i- XiSiZi — > Yi , UiSi — > ViXi — > Zi , 
UiSi Vi ~> Xi . 

Furthermore, we have 



i=l 



i=l 



I{Y'';KnMn) <y^I{Y,;VUA) 



i=l 



liZ'^; K„M„) < V I{Z,; VU,\S,) , 



i=l 



(43) 
(44) 
(45) 
(46) 

(47) 



/(if„;r"|Af„)-/(if„;Z"|M„) 

71 

1=1 

Proof of Lemma |6] is given in Appendix C. 

Proof of n^lV) C 7^i°"*'(^).• Let Q, X, Y, Z, S, U be 
the same random variables as those in the proof of TZ^ (F) C 
-^(^out)^P^ We further set V = USK„. Note that UVXSZ 
satisfies the following Markov chains 

U ^V ^ XSZ -^Y ,US ^VX ^ Z , 
US ^V X. 
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By Lemmas |2] and |6] we have 

Ro < min{/(r; US), I{Z- U\S)} + (Si,„ 
Ro + Ri< IiV;Y\US) 

+ min{/(r; US),I{Z; U\S)} + Jg.n 

< ^1 + ^4,11 

Re < I{V;Y\US) ~ I{V; Z\US) + S5^n . 



(48) 



By letting n 

e7^^•"(^). 



in ( |48] l. we conclude that (Rq, Ri, R^) 



C. Computation of Inner and Outer Bounds for the Gaussian 
Relay Channel 

In this subsection we prove Theorem |6] Let (^1,^2) be a 
zero mean Gaussian random vector with covariance S defined 
in Section V. By definition, we have 
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IN2 



2|1 : 



where ^2|i is a zero mean Gaussian random variable with 
variance (1 — p^)N2 and independent of ^1. We consider the 
Gaussian relay channel specified by E. For two input random 
variables X and S of this Gaussian relay channel, output 
random variables Y and Z are given by 

Y^X + S + ^i, 

z = x+e2 = ^Y + p,/^a+6ii- 

V J»i 

Define two sets of random variables by 

V{Pi,P2) = {{U,X,S) : E[X2] < P,,E[S^] < P2 , 

U ^ XS ^YZ} 

Vg{Pi,P2) ^ {{U, X, S) : U,X,S are zero mean 

Gaussian random variables. 
E[X2] < Pi,E[S'2] < P2, 
U ^ XS ^YZ} . 



Set 



A 



7^<™*'(Pl,P2|s) 

{{Ro, Ri, Rc) ■ Ro,Ri,Rc > 0, 

Ro < miii{/(r; US), I{Z; U\S)} , 
Ri < IiX;YZ\US) , 

Ro + Ri< I{XS;Y), 
Rc 1^ Ri , 

Rc < I{X;Y\ZUS) , 
for some ([/, X, S) G V{Pi,P2) ■} ■ 



A 



^f)(Fi,P2|S) 

{{Ro, Ri,Rc) ■ Ro, Ri,Rc^O , 
Ro < min{/(r; US), I{Z; U\S)} , 
Ri<I{X;Y\US), 
Rc 1^ Ri , 

Rc < I{X; Y\US) - I{X; Z\US) , 
for some {U, X, S) £ Vg{Pi,P2) •} ■ 



Then, we have the following. 

Theorem 7: For any Gaussian relay channel we have 

^(Fl,P2|S) C 7^d(Pl,P2|S) C ^(°"')(Pi,P2|I]). 

Proof: The first inclusion can be proved by a method quite 
similar to that in the case of discrete memoryless channels. 
The second inclusion can be proved by a method quite similar 
to that in the proof of Theorem |2] We omit the detail of the 
proof of those two inclusions. ■ 

It can be seen from Theorem [T] that to prove Theorem |6] it 
suffices to prove 



7^f^(Pl,P2|E) c7er(Pi,P2|s), 



J (out 



nr"{Pi,P2\^)^n[ 



(°"*)(Pl,P2|I]). 



(49) 
(50) 



Proof of (|49] | is straightforward. To prove 
preparation. Set 

„ ^ N2-PVN1N2 



, we need some 



JV1+JV2-2PVW1JV2 ■ 
Define random variables Y, ^1, and ^2 by 

dZ , 
6=11 



Y^aY- 

A 



(i-p')A'2«i+(JVi-pvivnv;)C2|i 



6 ^ 6 



Ni+N2~2p^/NiN2 
^V^)^l~^2|l- 



Let Ni = E[^?],i = 1,2. Then, by simple computation we 
can show that ^1 and ^2 are independent Gaussian random 
variables and 



(l-p^)AfiAf2 
N1+N2-2PVN1N2 



N2 = Ni+N2~ 2p^NiN2 . 
We have the following relations between Y, Y, and Z: 

Y = X + aS + ii, 

Y = Y + dS + ai2, 
Z = Y — aS — a^2 • 



(51) 



The following is a useful lemma to prove ( fSOl l. 

Lemma 7: Suppose that {U,X,S) g V{Pi,P2). Let X{s) 
be a random variable with a conditional distribution of X for 
given S = s. Ex(s)[] stands for the expectation with respect 
to the (conditional) distribution of X{s). Then, there exists a 
pair (a,/3) e [0,1]^ such that 

Es {Exis)X{S)Y ^aPi, 
h{Y\S) < ilog{(2^c)(aPi + A^i)} , 
h{Z\S) < ilog{(2^c)(«Pi + A^2)} , 
h{Y) < ilog{(27re)(Pi+P2 + 2x/^m + iVi)} , 

h{Y\US) = ilog{(2^e)(/3aPi +iVi)} , 

h{Y\US) > ilog{(2^e) (/3aPi +iVi)} , 

h{Z\US) > ilog{(2^e) {PaPi+N2)} . 
Proof of Lemma Q is given in Appendix F. Using this 
lemma, we can prove Theorem |6] 



13 



Proof of Theorem |6} We first prove ( |49l ). Choose ([/, 
X, S) e Vg such that 

E[X2] = Pi, E[52] = P2, 



^S + U, x = u + x, 



where U and X are zero mean Gaussian random variables with 
variance OrjPi and 9 Pi, respectively. The random variables 
X, S, U, and X are independent. For the above choice of 
(f7, X, S), we have 



Tfv- TTQ\ - n ePi+P2+2^§nPiP2 
l(Y ,Ub) I ePi+Wi 

/(Z; ells') = C ( 



/(X; y|[/S) = C j , I{X- Z\US) = C 

Thus, ( |49] l is proved. Next, we prove (l50l l. By Lemma [T] we 
have 

I{Y-US)^h{Y)-h{Y\US) 

^ r< ( (i-/3a)Pi+P2+2VaPiP2 ^ /iT^x 

- \ /3aPi+JVi J ' 

/(Z; U\S) ^ h{Z\S) - /i(Z|C/S') 



< C 



aPi 



^/3aPi+Af2 / ' 

IiXS;Y) ^h{Y)-hiY\XS) 

^ fj ^ (l-ga)Pi+P2+2VaPiP2 



I{X- Z\US) = h[Z\US) - 



> c 



( PaPl 
\ N2 



(53) 
(54) 
(55) 



I{X-YZ\US) = h{YZ\US) - h{YZ\XS) 
^ h{YZ\US)-h{YZ\XS) 
= h{Y\US) + h{Z\YUS) 

-h{Y\XS)-h{Z\YXS) 
= h{Y\US)-h{Y\XS) (56) 



C 



_§aPi 



(l-p^)iVliV2 
JVi + JV2-2p^]ViN2 



(57) 



where (1561 1 follows from 

h{Z\YUS) = XS) = h{Z\YS) 

- ilog{(27rc)a2iV2} . 
From ( I55] ) and ( |57] ). we have 



/3aPl 




(l-p^)iVlJV2 



- C ( ^ ) . (58) 



I{X;Y\ZUS) < C 

\ Nl + N2-3p\/"l"2 

Here we transform the variable pair G [0,1]^ into 

[r], 6) E [0, l]'^ in the following manner: 

a a " 



(3a, 7/ = 1 



1 - 



(59) 



This map is a bijection because from (|59|) , we have 

- ^ 
a = 1 - 6l?7 > /3 = - . (60) 
a 

Combining (|52li-(|54li, and we have (|50|. ■ 



Appendix 
A. Outline of Proof of Lemma Q] 
Let 

= {1,2,... , 2L"<'J } , £„ = {1, 2, . . . , 2L-i'"J } , 
J„ = {1,2,...,2L"'-^"'J}, 

where \_x\ stands for the integer part of x for x > 0. 
Furthermore, set 

W„ = {1,2,...,2L"'-'"'J}. 

We consider a transmission over B blocks, each with 
length n. For each i = 0, 1, . . . , i? — 1, let {wi,ti, ji, li) G 
W„x7^ xj7ii X £„ be a quadruple of messages to be trans- 
mitted at the ith block. For i = 0, the constant message 
vector {wo,tQ,jo,lQ) = (1,1,1,1) is transmitted. For fixed 
n, the rate triple ^ , rj"^ ^ , r^"^ ^ ) approaches 

{R^^\r["\ri"^) as S ^00. 

We use random codes for the proof. Fix a joint probability 
distribution of ([/, S, X, Y, Z): 

PusxYz{u,s,x,y,z) 
= Ps{s)Pu\s(u\s)px\us{x\u, s)T{y, z\x, s) , 

where U is an auxiliary random variable that stands for the 
information being carried by the message that to be sent to the 
receiver and the relay. In the following, we use Ae to denote 
the jointly e-typical set based on this distribution. A formal 
definition of A^ is in [15, Chapter 14.2], 

Random Codebook Generation: We generate a random 
code book by the following steps. 

1. Generate 2^"''' J i.i.d. s e 5" each with distribution 
Ui^iPsisi)- Index s(wj),Wi G >V„. 

2. For each s{'Wi), generate 2L"^o J i.i.d. u G Z^" each 
with distribution Yii=iPu{'Ui\si). Index u{wi,ti), ti G 

3. For each u{ti, Wi) and s{wi), generate 2L"''i J . 2L"''2 J 
i.i.d. X G A"" each with distribution ni=i Px{xi \si,Ui). 
Index x{wi,U, {wi,U Ji, k) G W„ xT„ x J'„ x 
r 

Random Partition of Codebook Tg. We define the map- 
ping (j) : Tn ^ Wn in the following manner. For each 
t aTn, choose w G Wn at random according to the uniform 
distribution over yV„ and map t to w. The random choice 
is independent for each t E %i. For each w G Wn, define 
%,iw) ^{te%,: (j}{t)=w}. 

Encoding: At the beginning of block i, let {ti,ji,li) be 
the new message triple to be sent from the sender in block 
i and (f;i„i, j,;„i, be the message triple to be sent from 
the sender in previous block i — 1. 

At the beginning of block i, the relay has decoded the 
message It then compute Wi — (p{ti-i) and send the 

codeword s{wi). 

Decoding: Let G y" and Zi G Z" be the sequences 
that the reviver and the relay obtain at the end of block i. 
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respectively. The decoding procedures at the end of block i 
are as follows. 

1. Decoder 2a at the Relay: The relay declares that the 
message ti is sent if there is a unique ti such that 

{s{wi),u{wi,ii),Zi) eAsuz,e, 

where Asuz,e is a projection of along with {U, S, Z), that 
is 

Asuz,e = {(s, M, z) e 5" X X Z" : 
{s,u,x,y,z) e Ae 
for some x,y £ X" x 3^"} . 

For projections of A^, similar definition and notations are used 
for other random variables. It can be shown that the decoding 
error e'^ in this step is small for sufficiently large n if 



4"^ < I{U;Z\S). 



(61) 



2. Decoder 2b at the Relay: For {w,t,j) e yV„ xTn xJn, 



set 



A 



t, j) = {x : X = x{w, t, j, I) for some / G £„} . 



The relay, having known Wi and ti, declares that the message 
ji is sent if there is a unique ji such that 

V{wi,ti,ji) n 

where 

^X\SUZ,e {s{Wi),u{Wi,U),Z,) 

= {xe A"" : {s{wi),u{wi,ii),x,Zi) e ^sc/Jfz,^} ■ 

(n) 

It can be shown that the decoding error e^]^ in this step is 
small for sufficiently large n if 

4"^ < IiX;Z\US). 

3. Decoders la and lb at the Receiver: The receiver de- 
clares that the message Wi is sent if there is a unique lii 
such that 

(s(Wj),yJ e AsY,e ■ 

in) 

It can be shown that the decoding error e\J in this step is 
small for sufficiently large n if 



r(") < I{Y;S). 



(62) 



The receiver, having known wi^i and Wi, declares that the 
message is sent if there is a unique such that 



and <Vi G Tniwi). 

(n) 

It can be shown that the decoding error e\-^ in this step is 
small for sufficiently large n if 



4"^ < IiY;U\S) + r 

<I{Y;U\S) + I{Y;S)=I{Y;US). (63) 



(") 



iwi.i,u.i,ii.i,ii-i) 

I 1 

I I 

XiWi.l,ti.lJi.],li.l) 



I ~ 

u(w;,r,0 



X(Wi,tiJ,li) 



ti-1- 



{Wi+l,ti+l,ji + l,li+l) 

I 1 



x{Wi+l,ti+j,ji+],li+l) 



ti+I . 



1 tl+1 



I ► |Dec.2b| ->i-j I — ► |Dec.2b| -> jj ' — ► |Dec.2b | -> ji+ 1 



I — »-| Dec.Ta]-^ 



V///////////777777\ 



1 



|Dec.lc| — 7;.2./,:-2 



to 



V///////////777777\ 



gbj-p- /,-2 - | ►iDec.lbl-p ti-1 - | ►iDec.lbly 

i JS; A ~\ i A ~^ 1 ^ A 



► I Dec. Ic I — 



Dec.la 



■ |Dec.lc| — 7,,/; 



Fig. 6. Encoding and decoding processes at the blocks i — 1, i, and « + 1. 

4. Decoder Ic at the Receiver: The receiver, having known 
Wi-i, ti-i, declares that the message pair h-i) is sent 

if there is a unique h-i) such that 

(^s{wi^i),u{wi-i,ii-i),x{wi-i,U-iJi_i,k-i),yi_i 

G AsuXY.e ■ 

(n) 

It can be shown that the decoding error ej^^ in this step is 
small for sufficiently large n if 



< I{X;Y\US). 



(64) 



For convenience we show the encoding and decoding pro- 
cesses at the blocks i, and i + 1 in Fig. |6l Summarizing 
the above argument, it can be shown that for each block 
i ~ 1,2, ■ ■ ■ , B — 1, there exists a sequence of code books 

{{s{wi),u{wi,ti), 
for n ~ 1, 2, • • • , such that 



1 



lim -log|7;i|= lim R^-'^ ^ min{I{Y;U S), I{Z;U\S)} 

n~^oo n n— i-oo 

lim - \og\Jn\ = lim r^"' =I{X;Z\US) (65) 
lim — log I Cn I = lini [r J" 

■ ■ — n^oo 



n^oo 77, n^oo n — >-oo 

IiX;Y\US) - I{X;Z\US) (66) 



lim 



'lb 



lim e^;') =0, 



lim eil> = lim e^^^ ^ . 

n — !-oo n~^OQ 



(67) 



Computation of Security Level: Suppose that T„, L„, J„ 
are random variables corresponding the messages to be trans- 
mitted at the block i. For simplicity of notations we omit the 
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suffix i indicating the block number in those random variables. 
For each block i = 1,2, ■ ■ ■ , B — 1, we estimate a lower 
bound of iJ(L„|Z"). Let Wn be a random variable over yV„ 
induced by <j> and the uniform random variable r„ over Tn 
corresponding to the messages to be transmitted at the block 
i - 1. Formally, Wn = (l>{fn). On lower bound of H{Ln\Z"), 
we have the following : 

H{Ln\Z") = H{LnJn\Z") - H{Jn\Z"Ln) 
> H{JMZ"TnW„) - H{Jn\Z") (68) 

By Fano's inequality, we have 



1 



1 



(69) 



The right member of ( I69b tends to zero as n — > oo. Hence, it 
suffices to evaluate a lower bound of i/(L„ J„|Z"T„W^„). On 
this lower bound we have the following chain of inequaUties: 

i?(J„L„|Z"VK„T„) 
= H{JnLnZ"\WnTn) - H{Z"\WnTn) 

= H{JnLn\WnTn) 

+H{Z^\WnTnJnLn) " iJ(Z"|T^„r„) 

= iog{|j-„||/:„|} 
+ij(z"|w^„T„j„i„) - ij(z"|w„r„) 

+H{Z''\W„T„JnL„) - H{Z''\WnTr,) . (70) 
We first estimate _ff (Z"| W„r„ J„L„). To this end we set 

A* = {iw,t,j,l,z) : 

{s{w),x{w,t,i,l),z) e AsXZ,e} 

By definition of A*, if {w,t,j,l,z) e A*, we have 
i logPZ"|X"S" {z\x{w, t, j, l),s{w)) 



-H{Z\XS) 



< 2e. 



Then, we have 



HiZ^lWnTrJnLn) 

> n[H{Z\XS) - 2e] Pr{(W^„, r„, J„, L„, Z") e X} 



>n[iJ(Z|XS')-2e](l-e, 



(n)^ 

2fc ) 



(71) 



Next, we derive an upper bound of W^„T„). To this end 

we set 



B* = {{w,t,z) : {s{w),u{w,t),z) e Asuz,,} 
By definition of B*, if {w,t,z) £ B* , we have 
1 



logp^„|y„s„(2|M(«;,t),sH)-ni/(Z|;75) 



< 2e. 



Then, we have 



if(z"|M/„r„) 

< n[H{Z\US) + 2e] + nKPr{(W„, T„, Z") ^ S*} 
<n[i/(Z|C/S') + 2e]+nKe^"\ (72) 



where k, = max 
-(|72]i, we have 



(s,«,2) l0g[pz|(7S 



, s) ^] . Combining d68] ) 



n 

>A^Hr^''^-I{X;Z\US) 

-46 - ^ - Ke':' - [r^ + i7(Z|X^)]e("J . (73) 



From ( |65] | -(|67]). and dTST l, we have 



lim -iJ(L„|Z") > I{X; Y\US) - I{X; Z\US) - 4e . 

n— »oo n 



Since e can be made arbitrary small, we have 



lim -H(i„|Z") > I{X;Y\US) - I{X;Z\US) . 

n— >oo 77 



For n = 1, 2, • • •, we choose block B = Bn so that 

-1/2 



B„ 



( r (^) (^i) (^)i 



(74) 



Define {g,Y,^^ by 



A 



if i mod 77, = , 



constant, otherwise . 



Then, we obtain the desired result for a sequence of block 
codes |(/„i3„,{5i}r=i"> V'«B„,¥'ni3„)| _ ■ Thus, the proof 
of Lemma [l] is completed. ■ 



B. Proof of Lemma |5] 

In the following bounding argument we frequently use 
equalities or data processing inequalities based on the fact that 
for i = 1, 2, • • • , 77, 5,; = g.i{ Z'-^) is a function of The 
notation [i] stands for {1, 2, • • • , n}~{i}. 

Proof of Lemma \3}; We first prove ( fTSl ). We have the 
following chain of inequalities: 

/(F";M„) 

= ij(y")-i?(y"|M„) 

n 

= ^ {i/(y,|r'-i) - H(r,|y^-iA'/„)} 

n 

< ^ {H{Y,) - ff(r,|y'-iz'-iM„)} 

n 

= J2 {H{Y^) - H{Y,\Y'-^Z'-^S,Mr,)) 

n 
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Next, we prove ( fT9] l. We have the following chain of inequal- 
ities: 

/(^";M„) 
= - ff(Z"|M„) 

71 

= {H{Z,\Z'-^) - H{Z,\Z''-HU)} 

n 

< {H{Z,\S,) - H{Z,\Y'-'Z'-^SM} 

n 

1=1 

Thirdly, we prove (|20] |. We have the following chain of 
inequalities: 

/(r";if„A.f„) 

71. 

n 

= J2 {HinY'-^) - H{Y,\Y'-^X")} (75) 

n 

< J2 {H{Y.) - H{Y,\Y'-^Sa")} 

n 

= J2 - H{Y, \XA)} (76) 

1=1 

n 

1=1 

where 

(ITST i: X" = fn{Kn, M„) and /„ is a one-to-one mapping. 

Y, ^ X,S^ ^ Y'-^X[^ '. 
Next, we prove ( l2Tl i. We have the following chain of inequal- 
ities: 

/(if„;r"|M„) 
<I{K^;Y"Z"\M„) 
= i7(r"Z"|Af„) - H{Y"Z"\KnM„) 
= H{Y'^Z'^\M„) - i?(y"Z"|X"M„) (77) 

n 
i=l 
n 

= J2 {H{Y,Z,\UA) ~ H{Y,Z^\UAX'')} 

i=l 
n 

= J2 {H{Y,Z^\UA) - H{Ya^\XA)} (78) 
j=i 

n 

< Y {H{Y,Z,\U,S,) - H{Ya^\U^XA)} 

n 

= Yl{X,-Y,Z,\U,S^), 

2 = 1 

where 

(iTTl i: X" = fn{Kn, Mn) and /„ is a one-to-one mapping. 



Finally, we prove ( |22] |. We have the following chain of 
inequalities: 

/(i^„;r"|M„)-/(A'„;Z"|A/„) 

< J(if„; F"Z"|A/„) - /(A'„; Z"|A/„) 
= /(A'„;r"|Z"A/„) 

= i/(r"|Z"A/„) - i/(y"|Z"if„Af„) 

= i?(r"|Z"A/„) - (79) 

n 

= ^ {F(y;|y'-^z"A/„) - ij(r,|y^-iz"x")} 

i=l 
n 

z^l 
n 

n 

= ^ {H{Y,\U,S,Z,) - i/(r,|5,Z,X,)} (80) 

i=l 
n 

< ^{ii(r,|[/,5,z,) - i/(r,|c/,5,z,x,)} 

i=l 
n 

= J2IiX^;Y,\Z,UA), 

i=l 

where 

(|79] l: X" = fn{Kn, M„) and /„ is a one-to-one mapping. 
Thus, the proof of Lemma |3] is completed. ■ 

C. Proof of Lemma \6\ 

The following is a key lemma to prove Lemma |6] 
Lemma 8: 

n 

liY-^; M„) < Y I{Y,;Yr+^Z'-^MnS,) , (81) 

n 

/(^"; MrO < Y ^(^^; Y,^^,Z'-'Mn\S,) , (82) 

i=l 

/(X„;r"|A//„)+/(y";Af„) 

n 

< ^ {/(i^„;r,|y,'tiZ*-iAf„^o 

i=l 

+/(K,;K/;iZ*-iA/„^,)} , (83) 
/(X„;y"|M„)+/(Z";A//„) 

< ^ {/(i^„; r,|y/;iZ*-iA/„^0 

i=l 

+/(Z,;y/;iZ*-iA/„|5,)} , (84) 
/(r";X„|Af„)-/(Z";A-„|A'/„) 

71 

= ^ {/(i^„; r,|r/;iZ'-iA./„5,) 
1=1 

-/(i^„;Z,|y,'^iZ'-iA/„5,)} . (85) 
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Lemma |6] immediately follows from the above lemma. We 
omit the detail. In the remaining part of this appendix we prove 
Lemma [8] 

Proof of Lemma \8}; We first prove dsTl i and ( |82] |. We have 
the following chains of inequalities: 

n 

= J2 {H{Y.\Y:U) H{Y,\Yr+M} 

n 

< ^ {H{Y,) - h{y,\y;i,z^-^sm} 

n 
i=l 

/(^";M„) 

n 

^ J2 {H{Z,\Z,_i) - H{Z,\Z,_iMn}} 

1=1 

n 

< {H{Z,\S,) - H{Z.,\Yr+,Z'-^S,Mn)} 

n 

= Y,I{Z,-Y^+^Z'-^Mr,\S,). 

Next, we prove ( [83] ). We have the following chain of inequal- 
ities: 

/(X„;y"|Af„)+/(y";Af„) 

= i/(y")-i/(y"|if„M„) 

71 

= ^ {H{Y.,\Y:],,) - H{Y,\Yl!,,KM) 

n 

< {H{Y,) - H{Y,\Y;:^,Z'-'S,KM} 

n 

= YIiy^■,Y^+lZ'-'s,KM 

n 

= Y {I{Y^;K,,\y:!,,Z'-^S,AU) 



i=l 



+I{Y,;Yr+,Z'-'SM} 



Finally, we prove ([84]) and ([85] l. We first observe the following 
two identities. 

n 

= J2 {H{Y,\Y,^^,Z''~HU) - H{Z,\Yr+,Z'-Hl,,)} ,(86) 

i=l 

H{Y''\KnMn) - H{Z'^\KrM 

n 

= Y{H{Y,\YIX,Z'^-'KM 



i=l 



H{Z,\Y;i,Z'-'KnMn)} 



(87) 



Those identities follow from an elementary computation based 
on the chain rule of entropy. The equality of ( [85] l immediately 
follows from (|86p — ([87]) . Now we proceed to the proof of 



( [84] ). We have the following chains of inequalities: 

/(X„;y"|A/„) + /(Z";M„) 
= i/(y"|A/„) - H{Y"\KnMn) + - i/(Z"|Af„) 

n 

= J2 {H{Y,\Y-^,Z^-'AU) - H{Z,\Y:1,Z^-^M^)} 

n 

+ J2{H{Z.\Z'-^) - H{Y,\Yr+,KM} 

n 

< Y {H{Y,\Y^^,z'-^sM - h{z,\y;i,z'-^sm} 

n 

+ Y{H{Z,\S,) - H{Y,\Y;:^,Z'-^S,K^Mn)} 

n 

= Y{nKn;Y,\Yr+,z^-'sM 

1=1 

+ IiZf,Yr+,Z'-Hu\S,)} . 
Thus, the proof of Lemma [8] is completed. ■ 

D. Proof of Lemma [5] 

In this appendix we prove Lemma H] We first present a 
lemma necessary to prove this lemma. 

Lemma 9: Suppose that /„ is a deterministic encoder. 
Set X" = fn{KnMn)- For any sequence {L^;}"^]^ of random 
variables, we have 



/(y"; i^„A/„) < Y I(Y^;X,UA) 



(88) 



/(Z";if„Af„)<^/(Z,;X,(7,|5,) (89) 

i=l 

Proof: We first prove ([88] l. We have the following chain of 
inequalities: 



I{Y^-KM 

H{Y^)-H{Y''\KnM,,) 

i7(y") ~i?(y"|x") 



(90) 



= Y {H{Y,\Y'-^) - ff(y,|y'-ix")} 

n 

< J2 {H{Y^) - i/(y,|y*-ix"5,)} 

n 

= ^{i/(yO-i^(y^|x,50} (9i) 

1=1 

n 

<^{ij(yO-i/(y^|x,[/,5,)} 

n 

= ^/(y,;X,(7,50, 



4=1 



where 

([got : X" = fn{Kn, Mn) and /„ is a one-to-one mapping. 

Y, ^ ^ Y'~^Xh] . 
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Next, we prove ( [89] l. We have the following chain of inequal- 
ities: 

J(Z";X„A/„) 
= i/(Z")-F(Z"|i^„M„) 

= - (92) 

n 

= ^ {H{Z,\Z'-') - H{Z,\Z'-^X^)} 

n 

= {H{Z,\Z'~') - 

n 

< {HiZ,\S,) - H{Z,\Z'-^X"S,)} 

n 

= Y {H{Z,\S,) - H{Z,XA)} (93) 

i=l 
n 

< Y {H{Z^S,) - H{Y,\X,UA)} 
1=1 

n 

= Y,IiZ^■,X,U^\S,), 
1=1 

where 

( [92] i: X" = fn{Kn, Mn) and /„ is a one-to-one mapping. 

m-- Z, ^ X,S, ^ Z'-^X^^ . 

Thus, the proof of Lemma |9] is completed. ■ 
Proof of Lemma^ Set Ui = Y^]_j^Z'-^ Mn- It can easily 
be verified that Ui, XiSiZt, Yi form a Markov chain Ui 
X^SrZ; Y^ in this order. From (HB, dSSi, and ^ in 
Lemma [8] we obtain 

n 

/(>^";Af„) <^/(y,;C/,5,), 

n 

/(Z";A/„) <^/(Z,;C/,|5,), 

1=1 

and 

/(r";i^„|M„)-/(Z";i^„|M„) 

n 

< ^{/(i^„; r.C/.5,) - I{Kn; Z,UA)} , (94) 

4=1 

respectively. From dSSl l. ( [89] l in Lemma |9] we obtain 

n 

1=1 

n 

I{Z^-KnMr,) <YliZ,;X,U,\S,), 

respectively. It remains to evaluate an upper bound of 

I{Kn-Y,UA) ^ I{K^-Z,UA) . 



We have the following chain of inequalities: 

/(i^„; Y,UA) - I{Kn; Z,UA) 
= H{Y,\UA) - H{Y,\KnMMS,) 

-H{Z,\UA) + H{Zi\KnMnU,S,) 
= HiY,\UA) - H{Y,\X^UA) 

-H{Z,\UA) + H{Z,\X'^UA) (95) 
^H{Y,\U,Si) 

-H{Y,\Za"UA) - I{Yf, Z,\X"UA) 

-H{Z,\UA) 

+H{Z,\Ya"U^S,) + /(y,; Z,\X"UA) 
= H{Yi\UA) - H{Y,\Z,X"UA) 

-H{Z,\UA) + HiZ.^X'^U.S,) 
^ H{Y,\U^Si) - H{Y,\Z,X,Si) 

-H{Z^\U^S^) + H {Z^\Y^X"U^S^) (96) 

< H{Y,\UA) - H{Y,\Z,X,UA) 

-H(Z,\UA) + H{Z,\Y,X,U^S,) 
= I{Y,; Z,X,\UA) - I{Zf, Y,X,\UA) 
= I{X,;Y,\UA)-HXf,Z,\UA), 

where 

( |95] i: X" = fn{Kn,Mn) and /„ is a one-to-one mapping. 

Z,X,S, -> U,X[,] . 
Thus, the proof of Lemma |4] is completed. ■ 



E. Proof of Lemma \5\ 

In this appendix we prove Lemma [5] 

Proof of Lemma Set t/i = F ^ ZJ!^ ^ M„ . It can easily 
be verified that Ui, XiSiZi, Yi form a Markov chain Ui 
XiSiZi Yi in this order. In a manner similar to the proof 
of Lemma [H we can derive the following two bounds 



/(i^"; M„) < Y I{Y^■,Y'-'z:XlMnS,) , (97) 

i=l 
n 

J(Z"; M„) < Y nZf,Y'-^Z^_^,M^\S,) . (98) 

i=l 

Hence, we have 

n 
n 

/(Z";M„) <^/(Z,;[/,|SO. 

2 = 1 

Furthermore, from dSSl l. ( [89] l in Lemma |9] we obtain 

n 
i=l 

/(Z";ir„M„) < ^/(Z,;X,i7,|5,), 

i=l 

respectively. It remains to evaluate an upper bound of 
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Since /„ is a deterministic, we have 

/(X„;y"|A/„)-/(A-„;Z"|M„) 
= H{Y"\Mr,) - H{Z"\M^) - H(Y"\X") 

. (99) 

We separately evaluate the following two quantities: 

H{Y"\Mn)-H{Z"\Mn), 
H{Y"\X")- H{Z"\X"). 

We observe the following two identities: 

ir(F"|M„)--ff(^"|Af„) 

n 

= E {H{Y,\Y'-^Z^_^M - H{Z,\Y'-^Zlt^,M„)} , (100) 

i=l 

n 

= {-HinYll.Z'^-^X'-') + H{Z,\Y,l,Z'-'X")} .(101) 



i=l 



Those identities follow from an elementary computation based 
on the chain rule of entropy. From (II 00b . we have 



(102) 



H{Y"\M,,)-H{Z^\M„) 

n 

= Y,{H{Y,\U,) - H{Z,\U,)} . 

1=1 

Next, we evaluate an upper bound of 

-HiY^lY^I^Z'-^X"^) + H{Z,\Y^1j^Z'-^X'') . 

Set ij, = Y;^^Z'-^X[h . We have the following chain of 
inequalities: 

-h{y^\y;^^z''-^x") + h{z,\y;x^z'-^x'^) 

= -H{Y,\X,Ui) + HiZ,\X,Ui) 

= -H{Y,\XAU,) + H{Z,\XAU,) 

= -H{Y,\Z,X,SA) + I{Y^■, Z,\XAU,) 

+H{Z,\Ya^S^U^) - I{Yf, Z,\XAU^) 

= -H{Y,\Z,X,S,U^) + H{Z,%XAU{) 
= ~H{Y,\Z,XA) + H{Z,\Y,X,S,U,) (103) 
< -H{Y,\Z,XA) + H{Z,%XA) 
= -H{Y,\X,Si) + m-Z,\XA) 
+H{Z,\XA)-I{Y,-Z,\XA) 
= -H{Y,\X,S,) + H{Z,\X,S,) , (104) 



where ( 1103b follows from Yi ZjXiSi — > Ui . Combining 
(fToTT i. (fToa . and (fT04] i. we obtain 

/(/i„; r"|Af„) - I{K„; Z^\M„) 

n 

<Y,{H{Y^\U,) - H{Z,\U,) 

i=l 

-H{Y,\XA) + H{Z,\X,Si)} 

n 

<Y,{H{Y^\Ui) - H{Z,\U,) 

i=l 

-H{Y,\XAUi) + H{Z,\XA)} 



= Y,{I{XA- Y,\U,) - I{XA- Z,\Ui) 

i=l 

+I{U,;Z,\X,Si)} 

n 

= Y,\U,S,) - I{X,- Z,\U,S,) 

i=l 

+C{Sr, U„ Y„ Zi) + I{U,- Z,\X,S,)} . 
Thus, the proof of Lemma |5] is completed. ■ 

F. Proof of Lemma [7| 

We first observe that by the Cauchy-Schwarz inequality we 
have 



y^E;,(s)X2(5)y^Ex(s)l)^ 



Es (Ejf(5)X(S')) < Eg 

^ EsEx(s)X2(5) < Pi . 

Then, there exits a e [0, 1] such that 

E5 (Ex(s)^(5))' =aPi. 

We derive an upper bound of h{Y). We have the following 
chain of inequalities: 

h{Y) 

< ilog{(2^e) (Exsl^ + ^l' + ^i)} 

= ilog{(2^e) (Ex|X|2 + 2Ejfs^5 + E55' 

< \ log{(27rc) (Pi + P2 + TE.xsXS + TVi)} . 

By the Cauchy-Schwarz inequality we have 

= E5 [5E;,(S)X(5)] 



(105) 



< v/Es^yEs {^x(s)X{S)) = /f^VS\ . (106) 
From ( 1105b and ( 1106b . we have 

h{Y) < i log { (27rc) (Pi + P2 + ^/5J\P^ +Ni)] . 

Next, we estimate an upper bound of h{Y\S). We have the 
following chain of inequalities: 



< Es 

= E5 



/i(r|5) = Es [M^(5) + 6)] 

:ilog{(2^e) (Vx(5) [X{S)] + N,)]] 
'\\og[{2^c)[^x(s)[X\S)] 

- {^x(s)X{S)f + N^)] 

< ilog{(2^c)(E5Ex(s)[^'(^)] 

-Es (Ex(s)^(5))' + A^i)} 

< ilog{(2^c)(aPi+iVi)} . 

Similarly, we obtain 

hiZ\S) < ilog{(2^e) (aPi+TVa)} , 

hiY\S) < ilog{(27rc) (aPi +iVi)} . (107) 
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Since 

h{Y\S) > h{Y\XS) = i log |(27rc)iVi} 

and dlOTt . there exists /3 G [0,1] such that 

h{Y\US) = ilog{(27re) (/3aPi + iVi) } . 

Finally, we derive lower bounds of h{Y\US) and h{Z\US). 
We recall the following relations between Y, Z, and Y: 

Y = Y + aS + a^2, (108) 
Z ^Y~aS-ai2. (109) 

Applying entropy power inequality to (1108b . we have 



_l_22h{Y\US} > _i_2'2h{Y\US) 
27rc ^ — 



2iro 

f3aPi 
(3aPi 



1 o2h(a£a 
27re^ 



(l-p^)jVlJV2 



Wi+A'2-2pVAfiiV2 
jVj' JVi N2-2pNi V jVi jV2 



JVi+7V2-2pVWiAr2 



/3aPi 
Hence, we have 

/i(>'|?7S') > ilog{(27re) (/3aPi +iVi)} . 
Applying entropy power inequality to (|109t , we have 



_J_22h{Z\US} > _l_22h{Y\US) 1 o^^faf^l 



27rc 



{l~p^}NiN2 



Nx+N2-2p^NiN2 
Nl+p^NiN2-2pNi^NiN2 
^ Ni+N2-2p^NiN2 



= (iaPi + N2 . 

Hence, we have 

h{Z\US) > \ log{(2^e) (/3aPi + TVa)} 
Thus the proof of Lemma [T] is completed. 
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